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Introduction 



i^ng the many curriculum programs that have been developed, there haa been 
little reaearch that has Inveatlgated the relatlonahlp betiteen teacher behaviors 
as prescribed by the curriculum developers and student outcomes such as achievement 
or attitudes. The research on teacher behaviors vlthln curriculum programs gener- 
ally falls Into tifo major categories. First, there are studies \Aiich describe 
curriculum specific teacher behavlora— thoae Inatructlenal activities or behavlora 
which are hypothesized to be laqtortant for the success of a given program— but do 
not relate these actlvltlea to atudent galna (for example, Ollvero, undated; 
Gallagher, 1966, 1968; Rats, 1968; Llndvall and Cox, 1970; Nledermeyer and 
Oalrymple, 1970). Second, there are atudlea which relate curriculum general teacher 
behaviors— those Instructional activities or behaviors which are hypothesised to be 
Important for the success of all or many programs— to student outcomes (for example, 

LaShler, 1967; Walberg, 1969; Flanders, 1970; Soar, 1971; Soar, Soar, and Ragosta, 

1/ 

1971). 

Unfortunately the results of especially the first group of studies can have 

limited Impact on the development or aaaessment of the teacher training prograsM 

vlthln particular curriculum packagea, or on the modification of the curriculum 

materials themselves. The descriptive studies, although suggesting wide variation 

In events within classrooms using a particular curriculum package, do not relate 

the variation to student outcome measures. For example, Gallagher (1966) counted 

various types of activities which occurred In the classrooms of six teachers who 

were teaching the same unit from the Biological Sciences Curriculum Study (BSCS) 

JL/ Some behavlora which are curriculum apedflc may also be curriculum general. 
That H, Inatruetlonal actlvltlea or behaviors which are Important for a par- 
tlculj^ program may be Important for other programs as well. The distinction 
between the two types of behaviors reflects two different research approaches. 
One research approach Is to study those behaviors which are Important across all 
or most teaching situations. Another research approach Is to study thosa 
behaviors which are In^rtant vlthln specific teaching situations. 
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program* On Almost all ne«sures of tfAchMT behavior there were eignif leant di££er- 
ences nong the six teachert. Regretably, the investigator did not relate this var- 
iation to measures of student outcomes. Does an increase in inquiry strategy 
behaviors which are intended by the BSCS curriculum planners enhance or suppress 
student achievement or is the effect negligible? Given a behavior that affects 
cognitive gains, what are the concoomitant effects in attitude towards the curriculum, 
towards the school, or towards the child? 

While the second group of studies do at'i:empt to relate instructional activities 
to measures of student outcomes, the observational instruments used were designed to 
apply to all types of programs and educational settings. For example. Soar (1971; 
Soar, Soar, and Regosta, 1971) has been monitoring eight classrooms in each of 
seven Follow-Through programs along with two comparison classrooms for each program. 

i 

Instead of developing program- specific observation instrtsnents Soar used four general 

2/ 

observational systems: the Reciprocal Category System (Ober, #61, an expanb m of 
the Flanders system, |5), the Florida Ttwonony of Cognitive Behaviors (K-1 Form/ 
(Browo. et al . #37), the Teacher Practices Observation Record (Brown, #36), anU the 
Florida Climate and Control System (Soar, 1966; Soar, Soar, and Ragosta, 1971). 
Although the investigators correlated the factor scores derived from the four instru- 
ments with measures of class mean residual 0ain, it is plausible that the most 
critical variables which affect student gains are those which were not included in 
the general observational instruments. The ability to follow a prespecified format 
without even minor deviations may be an Important variable in the Engeluann-Beclcer 
program, wherees in the Bank Street Program, the ability to elaborate on a child's 
experiences may be essential to the realisation of the program's goals and objectives. 
However, a general observation instrument is likely to be insensitive fo either of 
these progrma-specific variables. Therefore, in aitdition to general instruments, 
development of observational measures specific to the instructional activities most 
emphasised by the curriculum designers seems useful. 

2J Mmbers such as this refer to those assigned each observational system in 
o Mirrors for Behavior . (Simon and Boyer, 1967, 1970tt, 1970b). 

ERIC 
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Rosenshine and Furst (1973) suggeatad that research on a particular curriculum 
materials package should consist of £lve phases: 

1. Train a group of teachers to use a certain package of materials which have 
already received extensive trial and modification within special settings (for exople, 
any of the Follow-Through progrms like the Bank Street Program, Bushell's Behavior 
Analysis Progra, or Bngelmann and Becker's Dlstar Program> BSCS; First Year 
Communication Skills Program; or Harvard Project Physics). 

2. Use observational systems to describe Instructional variables which are 
considered specific to the program and most emphaslaed by the currlculua planners and 
which are also considered to have general educational Importance (and may or may not 
be emphasised by the curriculum designers). 

3. Study the relationship between Instructional activities and behavioral change 
In the students In a variety of outcomes. Problems and suggestions for selecting 
measures of Instructional behaviors and stu(!ent growth oh outcomes of Interest, and 
for data analysis and design are presented elsewhere (Hediey and Mlt«el, 1963; Gage, 
1969; Flanders, 1970; Rosenshlne, 1970, 1971; Rosenshlne and Furst, 1971, 1973; 
Tatsuoka, 1972). 

4. Modify the training procedures and/ or materials on the basis of the studies 
completed In phases two and three. 

5. Conduct new studies with appropriate control groups to determine the effects 
of the modifications and to detetmlne the new relationships between Instructional ac- 
tivities and student grotfth. By recycling through phases one through four, the currlc 
ulum designer, publisher, and researcher successively approximate optlmwi training pro 
cedures, thus affecting gains in student achievement or other outcomes of Interest. 

Although Rosenshlne and Furst* s ••descrlptlve-correlatlonal-experlmental-loop" 
design for curriculum research and evaluation is not unique (see Tatsuoka, 1972) , no 
study was found which included all phases of the design. Research studies which 
include part of the "loop" exist. However, even this type of instructional research 
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within curriculum progrant it rare. In fact, only two ttudiet were found which 
Included the training, deacriptive, and correlational phaaea md alao uaed prograa- 
apecific variablea: Kochendorfer (1967) and Baker (1969). 

Background of &tpef ^*>t"*'*^ ?*'V''T 
Deacription of the Plater Inatru ctionel Svatem 

One of the moat aucceaaful (Science Reaearch AaaocUtea, 1971a) and controver- 
aial of all the early childhood curriculum materiala programa la Diatar Reading, 
Language, and Arithmetic (Engelmann and Bruner, 1969, 1970; Engelmann and Camine, 

1969, 1970, 1972; Bngelmann, Oabom, and Bagelmann, 1969; Engelmann and Oabom, 

1970, 1972; Engelmann and Stem, 1972), a coai»erclal model of the Engelmann-Beeker 
(Bereiter-Engelmann) Pollow Through program. Unlike other progrwrned materiala, the 
Diatar program la not a •alf-inatruetional program. Inatead, the teacher followa a 
carefully atructured and logically aequenced teaching program. The preaentation 
booka provide the teacher with a acript, e aeriea of demonatrationa and teak* to be 
preaented word for word. The teacher* a role thua changea from one of deaignlng 
inatruction to one of teaching a particular format to criterion, involving all of 
the children in the inatruction, correcting mlatakea, providing feedback, and 
reinforcing the chil<iren*a reaponaea. 

Teacher Imr l'— t"*'*V ?P Variablea 

Five areas of teacher behavior are emphaaiaed throughout teacher guidea and 
training manuala: (1) followlag the format— uaing the exact wording provided in 
the materiala, not producing additional atatementa or aaking queatlona unleaa the 
format calla for them; (2) aignala— uaing clear aignala for the children to reapond 
ao that they all reapond at the aame time and not Imitate other children 'a reaponaea; 
(3) correctiona and criterion teaching— correcting children* a mlatakea aa they occur 

3/ For a mor^ complete outline of the philoaophy and methoda uaed in the Bagelmann- 
Becker program the reader la referred to Engelmann (1969a, 1969b) and Maccoby 
and Zellner (1970). 
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and requiring that they return to the beginning of the task to that they nay 
recognize each exercise as a series of steps iihich relates to a goal and to certain 
tales; <4) praise and feedback— reinforcing the children who are on task and 
relating their perfotaance to the rules of the task; and (5) pacing— noving at 
varying speeds through the lesson so that the children will understand the point of 
each task and so that their interest and enthusiasm will be maintained. 

According to the Distar curriculum authors, these^'are the basic implementation 
variables. It is assumed that if a teacher behaves in th«ie ways the children 
will achieve the academic objectives of the Oistar prbgrks. That is, the Dister 
curriculum developers hypothesize that the above teacher behaviors arc directly 
related to student achievement. 

Experimental Study 

Purpose 

The purpose of the experimental study was to determine (1) the feasibility of 
modifying the implementation level of a randomly selected group of Distar teachers, 
and (2) the effects of such training upon their students' achievement level. IWo 
related aspects of teacher behavior were chosen for experimental manipulation: 
correction procedures and criterion teaching. These variables were selected because 
(1) they «»re unique to the Distar program and have received consistent and statis- 
tically significant support in previous correlational studies (Slegel and Rosenshine, 
1973) ; and (2) they are the most difficult behaviors for most Distat teachers to 
implement appropriately. Establishing the Importance or lack of importance for 
these variables would greatly affect the design of future Distar training programs. 
Procedure 

Teachers and program. Fifty teachers from a large Southwestern school district 
received two days of in-service trai-dng before they began teaching Distar Language I 
(Engelmann, Osbom, and Engelmann, 1969). The program emjphasises the language of 
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Instruction— that is, the actusl language the teacher uses in the classroon— and 
tysteaatieally jtakes a child from identification of faniliar objects to the 
description and classification of those objects as well as concepts of logical 
reasoning. The training program, (SRA, 1971b) focused on teaching selected 
formats, analysing taks, correcting mistakes, rad general procedures fo** implement- 
ing the Oistar systsm. 

Students . The children in each classroom w^ire divided into three 'homogeneous*' 
groups based on a criterion-referenced, individually ateinistered pretest of 
language skills taught in the first 80 lessons of the program. One group from each 
classroom was randomly selected for further study. Each group consisted of five 
to ten first-grade children who had no previous experience with Distar. Forty-six 
percent of the children in the study were Mexican-Americans, 37 percent Black, 
and 17 percent Auglo. 

Collection gfl j t ?l*1"g of classroom observation data . Sach of the 50 teachers 
was audiotaped While teaching one lesson during each of six one-week se^ients. 
The six taping segments were equally spaced throughout a seven-month time period. 
The teachers did not know that they were to be taped until about five minutes 
prior to the teaching of a lesson. The research assistant turned on the cassette 
recorder, adjusted the volume, end left the room. She returned in 30 minutes to 
collect the tape. The teacher was m <er permitted to listen to the recording. 
For the most part, the content of the lessons for each taping occasion was rather 
uniform across teachers. 

The audiotapes were coded by three teans of two graduate students each. An 
observational systmn developed especially for coding any lesson in the Distar 
Instructional Systsm was used. The instrument was developed by Siegel and a team 
of Distar authors snd trainers. 

y S. Engeimana, J. Osbom, B. Bruner, P. Mshan, L. Meyers, D. Granat, and B. 
Rosenshine (consultant). 
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Interratcr •Krc<ment . Although all t«am members had previous experience ifith 
the Dlstar Instructional System Observation Instrawttt (DX80Z) , a system was 
devised to establish and to maintain consistency vithtn teams, since each team 
coded separate categories. Initially, members of each team coded five raiidamly 
selected tapes separately and then together to establish their consensus In 
Identifying and rating the behaviors specified by the InstnmMnt. Each team 
member then received a set of ten tapes to code. Upon completing the set, each 
tesm member randomly selected one tape for the other fm member to recode. After 
the ^.aded tapes ware independently coded for a second time, the tesm met to agree 
upon their ratings and, if necessary, to modify or extend the coding rules. This 
procedure was repeated until all 300 tapes were coded, thus providing 30 tapes 
which were used to detatmine inter judge correlations. For every category a 
judge* s score (rating or counting) for each task of a lesson was correlated with 
the other judge's srvxe for each task of the swe lesson. Table 1 presents the 
xange of inter judge correlations for each of the categories. 

Observation iastrment . Althou^ DISOI waa designed to be used with all these 
Dlstar progrmns (Language, Reading, and' Arithmetic) , a series of "ground rules" 
wera developed and ei^licltly stated so that consistent decisions could be made 
among coders while listening to audiotapes of teachers presenting the Language 
program. The following is a description of two categories of the Dlstar Instruc- 
tional System Observation Instrument. The entire Instrument is presented elseuhera 
(Slegel, 1973b). 

Corrections 

The paradlgp for correcting basic mistakes (according to the Dlstar 
Orientation Manual— Revised edition) is as follows: (Note: Repeating the 
entire task from the beginning will be considered a part of "criterion 
teaching") . 

Type I Mistake- -Lacks Information 

1. Teacher gives the answer (A) or provides additional information (Ai). 



2. TeachM t«sts th« child by rtpeAtlng th« ••0Mfit missed (T). 
Tyna II Mist Aa«-ttotor/Sn*iA Ptoblia 

1. T««chM gives ths anniwr (A). 

2. T««ch«r repMts the signtl (R). 

3. Teacher leads the child (L^). 

4. Teacher tests the child by repeating the seg^t aissed (T). 
T^na 111 tttsf k^-Poea Hot Uadeacstaiid Signal 

1. Teacher repeats the signal or calls attention to the sigfial (E). 

2. Teacher or another child aodels the response QO* 

3. Teecher tests the child ty repeating the segMnt aisaed (T). 
Definitions 

t^a anawar ik\ is sliiply telling the child the correct response. 

b. Providina additional infoiaatlon (Ail ia not telling the fWld the 
entire aaam but Mre^ providing extra InfonMtlon so that the child 
can "ceae up with** the correct reaposae. 



c. T>atlna the chUf m- Aildran CT) ia asking the <|ueation again or requiring 
the cbild(ren) to respond. 

d. Leading (11.) is responding idt^ the child. The teacher and the child 
aWtan^a ly say the reaponae. The **n** Indicates hov mmcf tlMS the 
teacher Tays the response with the child. Ideally the teacher should 
lead tifo or nore tiaea. 

e. Btfttstititr fhf tflTfl (?) i« identical to teating the child. The teacher 
IlkTtfcrqStliraUto o ^ repeats the cciand to respond. The difference 
In labelling Is a function of nben the question ia reaskad. If it appeara 
aa the first step in the paradiflp, then it ia <R); if the question is 
repeated as the last atep in the paradlgp, then it la (T). Monaiiy 

the teacher does not expect the child to correctly answer when she 
**repeats the aignal.** The purpoae of this step is aerely to call 
attention to uhat the children should be responding to. 

f. Modallna (Xi Is perfoialng the teacher's part and the child's part. 
This is done to d^onatrate to the child how the tiio parts are related. 
The teacher aaks the question (or requires a response) and then answers 
the question (or perfotas the response). Mote that in (A) the teacher 
only answers the question (or perforas the correct response). 

This category is coded as follows: 

A. Bach alstake is classified as either Type I, 11* or III. 
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B. The teacher's correction procedure is indicated by using the ebove key* 

.^Mle: *'R M T*' ipould indicate that the teacher repeated the signal, 
nodeled the response, and tested Che child on the ses&ent nissed. 

SSSaele: *'R L3" would indicate that the teacher repeated the signal and 
led the response three times. 

C. Each correction is rated according to the following eight-point rating 
acheae: 

8: Teacher corrects the mistake iamediately after iw occurs. 

If the mistake is Type I (lacks information) , teacher gives the 
answer (A) or provides information (Ai); 

If the mistake is Type II (motor/speech) , teacher gives repetition 
activities (L^) where n is greater than or equal to two. (The 
teacher may proceed the leading with giving the answer (A) and/or 
repeating the signal (R).); 

If the mistake is Type III (does not understand the signal) , teacher 
repeats the signal or calls attention to the signal (R) jnd teacher 
or another student provides a model (M) ; 

then the teacher tests the child (or group) by repeating the segpent 
of the task that was missed (T) . 

irrtrrU f T^ ^vpe I mistakes ! AI 

EMmples for Type II mistakeai L2T; L^T; AL^T; RAL^T 

V^h* Type III mistakes ; RHT 

7: Teacher pairs the type of mistake with the correct procedure, as 
indicated above but adds additional procedures gsl tests. 

^HT^Tf Type I mistakes ; MX (note that A is included in M) ; 
AB (note that T is included in B— repeating the entire task irom the 
beginning); ML^T; RAT 

EMPles for Type II mistakes t MLjT; RAML3T 

Examoles for Type III mistakes ; ARML3T; RL^MT 

6: Teacher pairs with type of mistake en incorrect procedure 
(including omitting steps) jod tests. 

EKamoles for Type I mistakes ; LjT; RLj^T 

T*ffT?fT "O'f II mistakes ; AT; RMI 

ftiiaples for Type III mistakes ; MX; AT; UT; AL^T 
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5: Tetts only (T) or rept«t« the eatirt task only (B). 

ftffffflft fa^ ^ ulatakfti ; T; B 

Ek«dUi for Type II «lif k«a ; B; T 

BkmoIm for Typ« I^ «litak«« t T; B 
4: Like 8 but with no tost. 
3: Liko 7 but with ng tost. 
2: Like 6 but with no tost. 

I: Mo corroction «t oil or fivlng tho wrong answor or InfotMtlon. 
Crltorton T— chinK 

This bohsvlor is countod wbtnovtr it occurs. Notice thet the behsndor 
aey or mey not follow en eppropriete correction procedure. Thet is, this 
cetegory is independent of the reting received for cetegory II » corrections. 

A. For Type I aistekes, the teacher repeats the entire teak from the 
beginning (Bl). 

B. For Type II niatakea, the teacher repeata the entire taak fron the 
beginning (B2). 

C. For Type III aiatakea, the teacher repeata the entire taak frca the 
beginning (B3). 

0. For Type I, II, or III aiatakea, the teacher repeata a feanent of the 
taak but not the entire taak (8). A ae»ent of a taak conaiata of 
repeating one or nore aipials prior to the aignal miaaed— but not from 
the beginning. 

B. The teacher recyclea through (repeata) a aaffeent of the taak or the 

cooplate taak (BC). Ihia behavior differa froa AO in that the recycling 
doea not inedUtely follow a niatake. The aaquence ia aa followa: 
(1) a child nakea a viatake; (2) the teacher may ot mtf net correct the 
niatake (appropriately) ; (3) the teacher proceeda with the next part of 
the taak; and (4) the teacher recyclea through a portion of the taak or 
the entire task. 

Althouib the prlaary intereat of thia atudy waa to determine the rele- 
tionships between correction procedures, criterion teaching and atudent 
echievement (veriablea 4-7; 9-15) data waa collecced for other variablea aa 
well. Thia enabled an exnination of poaaible changea in other behaviora aa 
a function of training in correction procedure and criterion teaching behav- 
iora. Rating aheeta were uaed to code the data. For each of the 27 variablea 
the total acora waa divided by the total number of taaka (or total number of 
miatakea, in the caae of correctiona and criterion teaching) which occurred in 
a given 30-minute taping. 
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Selection of experimmf 1 and control group teachers . After the first three 

observations were coded, the teachers were divided into "high" and "low" Implenen- 

tors according to their average scores on variable 7 (CT) and IS (BT + S + RC)— 

5/ 

correction-total mistakes and criterion teaching, respectively. 

The high and low Inplementors were then randomly divided into two groups of 
25 teachers each. (See Figure 1) One group was randonly designated the experimental 
group, and these teichers were invited to attend a one-day retraining workshop. 
Unfortunately only 23 of the 25 invited teachers attended the retraining session. 
The two teachers that did not attend were high implementors. Their principals 
explained that they were already excellent teachers and did not need retralaiag. 
Therefore, they were added to the control group (27 teachers). This probably 
biased any possible outcomes in favor of the control group. 

Second pretesting of students . Prior to the retraining workshop, a mid-year 
pretest was administered by a trained research assistant to all children in the 
experimental and control groups. The measure used was the Continuou s Test for 
Language One (CTL) (Bngelmann-Becker Corporation, 1971). This test is use^ by 
the Engelmann-Becker Follow Through Model to measure pupil performance in the 
Oistar Language program. The oral test is criterion-referenced and individually 
administered. 

The CTL consists of several subtests which measure mastery of each major skill 

taught in the level I Language program: object concepts, action statements, p4rt«, 

sentence repetition, prepositions, categories, plurals, pronouns, function words, 

verb tense, comparatives-superlatives, if-then, before-after, only. For the 

purposes of the pretest, one- third of the 114 items was administered. These items 

5/ More specifically, principal^oomponents analysis was used to determine two 
linear composite scores for each teacher. The rank order of each teacher's 
sum of composite scores then indicated her degree of implementation. 
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Included concepts from lessoas 24 to 93. (There are 180 lessont in the Language I 
program.) 

An example of two items appears in Figure 2. If an item had several parts, 

all parts must have been correct for the item to be scored "pass." 

Description of treatment . The experimental teachers were provided with one day 

of retraining on the behaviors of "correction procedures" (variables 4-7) and 

V 

"criterion teaching" (variables 9-15). During the week following the workshop, 
each teacher also received 30 minutes of classroom consultation. The trainer 
concentrated on the above two sets of behaviors while observing the teacher teach 
a lesson. The control teachers received no additional training* nor did they 
receive classroom consultation. 

The one day retraining workshop (Siegel, 1973a) focused on several of the 
teaching problems which were noted during the coding of each teacher's three 
pretreatment audiotapes. These were as follows: 

a. distinguishing an appropriate response from a mistake; 

b. distinguishing different types of mistakes; 

c. correcting a mistake according to the type of mistake; 

d. correcting a mistake even though several children in the group responded 
appropriately; 

e. testing the child or children on the segment of the Usk that was 
Incorrect; and 

f . returning to the beginning of a task after a mistake was corrected. 

A simplified three- step correction paradipn was presented during the retraining 
session (see Figure 3). 

6/ The reliability of the 38 items for all children in the experimental and 
control groups was found to be .89, using KR20. 

2/ In « strict sense, criterion teaching— returning to the beginning of a task 
after a mistake has been corrected— is reflected only in variables 9-12 
(El, E2, E3, ET, respectively). However, a less conservative interpretation of 
"criterion teaching" would also include variable 13 (returning to a segment 
of a task after a mistake has been corrected) , variable 14 (recycling through 
the task for "firm-up" purposes) and variable 15 (the total of variables 12-14). 
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Do something that will praVlmt the mistake £rom occurring again: 


If the child lacks 


If the child has a 


If the child does 


information: 


motor or speech 


not understand the 




problem: 


signal: 


Tell the answar. 


Tell the answer. 


Repeat the siftnal. 






(or call attention 




Sav the snswer with 


to the signal) 




the child 3 to 6 






times. 


Redo the sement 






with vou tellimt 






the snswer. 



Redo the segment of the task that was incorrect . 



Redo the task from the beginning with the entire group . 



Note : If the mistake was a motor or speech problem, it will be Impossible 
to teach to criterion in one day. At step thrsa, we therefore 
expect only incremental improvement. The goal in correcting other 
types of mistakes is that at step 3, all the children in the group 
can perform the entire task- -from start to finish- -without making a 
mistake. 



Figure 3. Three-Step Correction Paradigm used in Retraining Workshop. 
(Siegel, 1973, p. 33) 
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Poittestlng of «tud«nt« . All children were posttested on the following 
measures: 

1. The Engelmann-Becker Continuous Test for Leneuege One (Engelaann-Becker 
Corporation, 1971); and 

2. The APgLL Test (Bdcodyne Corporation, 1971), 

Both measures were Individually administered by a trained reseat ch assistant. 
Fif ty-two items were selected from the larger llA-ltem C ontinuous Test , spanning 
lessons 44 through 133. The APBLL Test consisted of 23 items and all were given. 
The reliability of these measures (KR20) was .92 for the CTL and .66 for the APBLL. 

The APELL Test represented a measure of language achievement which was not 
specifically developed to test outcomes of the Distar Language program (although 
there were overlapping objectives between the test and the progr«i). The measure 
waa criterion-refertoced and assessed six areas of language developmeiit: nouns, 
pronouns, verbs, adjectives, plurals, and prepositions* Bach item of the APKLL 
Test consisted of three pictures. The child was asked to touch one of the three 
pictures. For example, item 22 displayed a picture of a cup under a table, a 
picture of two cups on a table, and a picture of a cup on a table* The tester 
said, "Look at the pictures. Put your finger where there is nothing on the table ." 

Results 

There were actually two main parts to the question, "Did the treatment work?'*" 
The first was whether there were significant differences between the experimental 
and control group of teachers in behavior on the variables of correction procedures 
and/or criterion teaching. The second was whether there were significsnt differences 
in adjusted achievement on the students' posttests i^etween the experimental and 
control group of teachers. Each part was further subdivided into two additional 
questions: were there differential and interaction effects for the high and the 
lew implementing teachers? 
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Amilytlt of Teacher Behavior 

Multivariate analysis of the retraining variables * A ••mixed" or •» split-plot" 
NANOVA design was used to answer the question of significant modification of 
(changes in) behavior. The design had two between-block treatments (A) and (C) 
and one within-block treatment (B) . Treatment A had two levels— experimental and 
control; treatment C had two levels— high and low implementation on correction 
procedures and criterion teaching. Treatment B also had two levels— the mean 
score of audiotaping observations 1-3 and the mean score of audiotaping observations 
4-6. That is, both sets of observations were treated as dependent variables 
obtained at two different levels of time factor— one prior to the treatment and one 
subsequent to the treatment. 

Table 2 presents the repeated measure MANOVA for the 11 retraining variables. 
The siginlf icatn (£ <.66l) Treatment by Occasion (AB) Interaction and Inspection 
of the gt%up centrolds indicated that the experimental group after the retraining 
session performed significantly higher than the control (no retraining) group. 
There were no significant differences (£> .05) in implementation before the 
retraining. Discriminant analysis furthermore revealed that the criterion teaching 
(ET) variable— repeating the entire task after a mistake has been corrected— 
contributed most to the dimension along which the difference between the 
experimental and control group's ••post minus pretreatment'^ difference was 
maximised. 

Univariate analyses of retraining and nonretralnlmt variables . For each 
of the correction procedure and criterion teaching variables, the treatment group 
by occasion interaction (AB) effect was significant (£ <.05). Furthermore, a 
posteriori comparisons among group means, using Tukey*s HSD test, revealed that 
for each of the 11 retraining variables, the experimental group teachers after 
retraining were rated significantly (£<.05) higher implementors than the control 
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TABLB 2 

R«pMted IfoMurea H480VA for the 11 Ratralnlng Varl«ble« 



Source of VerlAtion 


Degree* of Freedom 


F-Retio for Hultlvarlate 
left of Iquellty of 
Keen Veetore 


Treetaent (A) 


11/36 


5.290*** 


IiVleneneeelon <C) 


11/36 


3.395** 


A X C 


11/36 


0.750 


Occeeion (B) 


11/36 


13.588*** 


A X B 


11/36 


9.186*** 


B X C 


11/36 


1.955 


A X B X C 


11/36 


1.241 



*£ <.05 

**£ < .01 
***£ < .001 
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(no retrUnlng) group teachers. Although there weru no eignlf leant differences 
between the ratings of the two groups prior to the treatment. Table 3 presents 
the P - ratios and probabilities for the 11 retraining variables. Figure 4 
Illustrates for the total criterion teaching score (variable 12) the treataent by 
occasion Interaction at each level of Implementation. 

Palrwlse comparisons of the AB group means also Indicated that the teachers 
In the experimental group significantly (£^.05) Improved In performance from the 
first three taping occasions to the second three taping occasions. This result 
waa found %rlth each of the 11 variables, whereas for the control group teachers* 
significant improvement In performance over the two sets of taping occasions 
occurred only with variables 4, 7, 13, and 15 (see Table 3). However, even though 
the control group teachers significantly Improved over time on four of the variables, 
the experimental teachers not only made significant improvements over time, but 
these laqprovementa were also significantly greater than the control teachers on 
all variables! 

Contrary to what might be expected, the retraining program did not differentially 
affect the high and low imptementors of the experimental group. That la, the hi^ 
and low experimental teachers made "equal" gains in implementation (see for exnple 
Figure 3). 

Although only the correction procedure and criterion teaching variables were 
specifically emphasised during the retraining of the experimental teachers, 
analyses for nonretralnlng variables are presented in Table 4. Of course, caution 
should be taken in interpreting the results of a series of separate univariate 
analyses. 

Perhaps the most interesting result of the analysis of nonretralnlng 
variables was the experimental group's significant Increase over time, as conq>ared 
to the control group's performance, on variables "repeating the correct response 
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BXPERIMSNIAL GROUP— HIGH IMPLBMEIIIORS 



CONTROL GROOP— HIGH IMPLEMENTORS 



EXPERIMBNTiO. GROUP— LOW IMPLBMEM10RS 



O- O CONTROL CROUP— LOW IMPLEMENTORS ' 




1 



MEAN PRBIREATMENT 
OBSERVATIONS 



1 



OBSERVATIONS 



MEAN POSTTREATMENT 
OBSERVATIONS 

(»,) 



Plgure 4. Tr«atm«nt (A) by Observation (B) inUraction at each laval of 

implntntatlon (C) for variabla 12 (critarlon taaehlng following 
all aistaket). 
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(21)," "general usk praise (22)," and "specific behavior praise (25)." This 
result provides iaportitat counter-evidence to those teachers and administrators who 
believe that teachers who teach to criterion or require maotery perfomance from 
their students tend to focus on negative aspects of student perfomance— thus 
making the Instructional setting averslve to the students. Quite to the contrary, 
the results of the present study suggest that the experimental (criterion teaching) 
teachers praised more often (with general and specific words of praise) than control 
(noncrlterlon teaching) teachers. 
Analysis of Student Achlevep ent 

The second major question was whether there were significant differences in 
achievement on the students* posttest scores after they were adjusted statistically 
for the differences on the pretest score. The technique of multivariate analysis 
of covarlance and discriminant analysis served to answer this question. 

A tw^-way MANCOVA design was used. Factor A had two levels (experimental and 
control) and Factor B also had two levels (high and low t4aplementors on correction 
procedures and criterion teaching). The dependent variables were (1) the 
Continuous Test for Lfln<^ui|B« (CTL) and (2) the APELL Test . Together these 
tests reflected ''classroom and Instructional language competency." The covarlate 
used In the analysis was the Pre-Contlnuous Test for Language One (PCTLV. This 
measure was administered one week prior to the treatment. Mean scores for the raw 
score and adjusted dependent achievement measures and covarlate are presented In 
Table S. 

In addition to the usual multivariate normality assumptions, the conditions 
which the data must satisfy If the re»»ults of the analysis iof covnriance ftre to have 
suitable generality are (1) the regression plane of the posttest score, on the 
pretest scores in the experimental group must be parallel to that In the control 
group, and (2) residual variation about the regression planes for the two groups 
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TABLB 5 

Unadjusted Kid Adjusted Mean Pretest snd Posttest Scores 
Used in the Multivariate Analysis o£ Covarlance 



High Low All 

Inplementors Impleaentors Inolementors 

Experlnental 

Group 20.916 16.449 18.585 

Group 20.619 21.456 21.022 



Bxperlnental 27.897 21.820 24.726 

Group AdJ: 26.574 AdJ: 25.670 AdJ: 26.122 

g| Control 24.750 24.908 24.826 

. Group AdJ: 23,891 AdJ: 22.988 AdJ: 23.M 



fccperinental 22.808 21.488 22.119 

Group AdJ: 22.478 AdJ: 22.328 AdJ: 22.400 



Control 21.680 21.845 21.759 

Group AdJ; 21.552 AdJ: 21.402 AdJ: 21.480 



Note: Standard deviations for the adjusted OIL and APELL are 4.706 and 1*318, 
respectively. 
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must b« homogeneous. Tests of these hypotheses Indlcsted thst the assumptions were 
tenable (£>.50). 

The results of the multlvarlete analysis of covarlance for the main effect due 
to treatment (retraining or no retraining) are presented in Table 6. The significant 
F-test (£<.05) leads us to the conclusion that there was information in the lan- 
guage measures (CTL and APELL) that was not in the pretest measure (FCTL) regarding 
differences between the retrained teachers (escperimental group) and the nonretrained 
teachers (control group). The correlations between the adjusted criterion variables 
and the discriminant function (factor structure) indicate how much each dependent 
variable is contributing to the discrimination among the levels of the treatment 
factor. Since the APELL Test has the highest correlation (.94) with the discriminant 
function, it appears that the discrimination among the groups is— to a somewhat 
greater extent— due to differences in scores on the APELL (the progrn-general test) 
rather than to the CTL (the program-specific test) . As can be seen from the mean 
discriminant scores of the two .treatment groups on the discriminant function, the 
experimental group received the higher mean discriminant score (26.088). 

The results of the multivariate analysis of covariance for the main effects due 
to implementation indicated tiiat the effect vas not significant (£-ratio for the 
muLtivariate test of equality of mean vectors ■ 0.172, £>.84). This means that thA 
treatment was equally effective for the high and low implementors. Likewise, the 
interaction effect was not significant (F » 0.062, £>.94). 
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TABLB 6 



Results of Multivsriattt Analysis o£ Covariance 
for Main Bf facts Oua to Traatment 



F-Ratio for Multivarlata Test of 
Bflualltv of Mean Vectors 



df hypothesis 2 

df error 44 

F 3.309 

£ < .046 



Standardised Oiscrlninant Function 
Coefficients 



CTL .405 
APELL .742 



Factor Structure for the 
Discrininant Function 

CTL .76 
APSLL .94 



Experimental Group 
Control Group 



Mean Disc <'<«»»<«]flpy ?*r**rif 



26.088 
24.774 
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CONCLUSI0N8 

Siamnarv of Remlti 

On« "pMs" through the descrlptlve-corr«latlonal-exp6rln8&tal-loop resMreh 
par«dl0a h«s baen completed with the Dieter progrem. An obeervatlonel Inetrument 
wee developed %fhlch reflected the specific teecher behevlors that the progrcn 
developers believed to be Importent for student cognitive gain. In two snail 
studies (Slegel and Rosenshlne, 1973) It was determined thet teacher behaviors that 
were considered Importaet for s'lccessful program Inplsnentetlon (following the 
format, approprlete correction procedures, teechlng e format to criterion, 
requiring unison responding to signals) were significantly releted to student 
achlev«Bent. Furthermore, the present study provided experimentel supi^brt for 
specific correction procedures and criterion teeching— two categories of behaviors 
which were charecteristic of the Dlstar Instructional System. A randomly selected 
group of high end low Implementing teachers Tfere retrained in techniques of 
correcting students' mistakes according to a prespeclfled paradl^ and recycling 
throu^ an instructional tesk until ell of the children in the group respond 
without error. As e result of retraining, the experimentel teachers performed et a 
elgnlflcantly hl^er level of implementation than the control teachers. In 
addition there were significant differences (fevering the experimental group) in 
language achievement on the students' posttest scores efter they were stetisticelly 
adjusted for the differences on the pretest scores. Thus, significant changes 
in teacher behavior (and partlculerly elong the dimension of criterion teechlng— 
the behevlor of repeating the entire tesk efter a mistake hes been corrected) 
epperently ceused significant changes in student achievement (on both e program- 
specific and program-generel measure). 

However, even though the differences in echlevement were significant, the 
differences were not es great as might be expected from the substantial modifications 
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In teacher behavior. Large changes in teacher Implementation did not cause large 
changes In student achievement. One could therefore reasonably ask whether it la 
vorth the effort (time, money, inconvenience) to retrain teachers with the result 
of small (although statistically significant) gains in student achievement. 
Unless one is to be misled, the answer must go beyond the normative or comparative 
results of training. For exsiDple, the experimental group's mean criterion teaching 
(£T) score was 2.4 standard deviation units above the control group's mean score 
for the seme variable. Nevertheless, the experimental group repeated the task from 
the beginning, after a mistake was appropriately or Inappropriately corrected, only 
20 percent of the time; the control group recycled to the beginning of the task 
following 2 percent of the corrected mlAtakes. In an absolute sense the experi- 
mental group was teaching to criterion at a low-moderate level of implementation, 
at best. Statistically significant differences which account for a large percentage 
of the variance may not be educationally significtfit. Justification of retraining, 
therefore, should be viewed in terms of absolute standards of performance (as well 
as normative stenderds of performance). 
A General Research Strategy 

The descriptive-correlational-experimental-loop paradigm is perhaps most 
readily understood within the context of a behaviorally oriented or structured 
curriculum prbgrsm such as the pistar Instructional Systsm. Yet it is proposed that 
this research strategy would be applicable to less structured and more "open" 
curricula as well. 

Table 7 illustrates this application and suggests various progrmn specific 

(important) process variables and possible outcome variables for three early 

childhood progrsms. The program implementation (process) variables and outcome 

8/ Although the description of variables for each progrsm was inferred from the 
literature, progrsm authors for each curriculum may disagree with the wording 
or emphasis in this table. However, the purpose here is to illustrate the 
versatility of the deseriptive-correlational-experimental-loop research paradiys 
and not necessarily to give an accurate description of process and outcome 
variables for specific programs. 
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varlables are priaarlly deterained by the curriculum developers and nay or nay not 
be stated explicitly. Furthemore, the behaviors which are enphasised during 
preservice and in-service teacher training nay vary from what is expressed in the 
writings (Journal articles, books, teacher guides, *»tc.) of the curriculum design- 
ers. Nevertheless, the researcher and program developer should ultlnately concur 
on the implenentation variables and on the procedures and instrunents used for 
collecting teacher and pupil data. If this were not the case, then the program 
developer could argue— Justifiably— that the researcher's study did not test the 
program's implenentation variables. It would be rather difficult for a researcher 
to Justify the inclusion of a particular "progrm- specific (Inportant)" variable 
on a progran- specific observation instrunent when this clain is denied by the 
program authors. A conpromlse solution, however, is possible. The observation 
instriment could reflect three types of variables: those varibles which the 
curricultm developers and researchers hypothesise to be Inportant for the success 
of the progran, those variables which only the curriculun developers hypothesise 
to be Inportant, and those variables which only the researchers hypothesise to 
be inportant. Although this would not obviate the above situation, the research 
would reflect each group's biases. This procedure could be expanded to include 
Inplenentation variables which are specific or Inportant to other curricula, as 
well as variables (especially outcone variables) which are inportant to different 
groups (parents, educators, legislators, students, etc.). For exanple, Amlngton 
would be concerned with neasures of curiosity and inagination for children in 
the EDO Progran whereas nany parents nay be concerned with neasures of reading 
and arithmetic achievenent. Both sets of variables nay be collected— not only 
for the EDO Progran but for other prograns as well. 

The research paradign* furthemore, does not specify the nanner in which 
inplenentation and outccno variables are to be collected. Perhaps observation 
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instrumtnts irauld be us«d— or possibly student r stings, teacher or parent 
questionnaires, video taplngs, audio tapings, surveys, or a series of "unobtrusive" 
measures. In fact, certain programs will typically value one form of data 
collection over another. For exmsple, the Engelmann-Becker program would be 
satisfied with norm-referenced and criterion-referenced tests of achievement. Ibe 
EOC Program, on the other hand, would probably value more Indirect measures of 
student behavior and attitude. Again, many types of. measures could be used for 
each program If more generallsable relationships are of Interest. 
Soma Implications for Teacher Bducatlon 

A major concern, previously implied in the introduction to this study, is 
whether or not generalised teaching behaviors are of importance to the imploenta- 
tion of curriculum progrmns. It was suggested that teacher behaviors which 
are specific to a program may be more important, in affecting student outcomes than 
teacher behaviors which apply to a wide range of progrsms. Indeed, no teacher 
teaches the curriculum "first-grade reading;" ^e teaches the SWRL Reading 
Progrn, the Bank Street Readers, IPI, Addlson-Wesley, Distar Reading, McGraw- 
Hill, or SRA Raading Labs, for example. Teecher behaviors which are critical to 
the success of one progrmn may not be very important to the success of another— or 
mtiy even be detrimental. 

Granted, there are similarities between progms. For exampU, a teacher 

surely would not have to learn a new set of teaching skills when she teaches 

Distar Reading after having taught Distar Language. Likewise, there are certain 

similarities between programs orlanted toward cognitive growth, between curricula 

oriented toward behavior modification, and so on* However, it is hypothesised 

that as the classification becomes more encompassing— goes beyond, a apedflc 

currleuluB program to Include programs of a general type and ultimately, models 

of instruction— the probability d ecreases that any teacher behavior that applies 

2/ Curriculum program is here defined in the broadest sense— ranging from a 
published set of materials to a philosophy or model of instruction. 
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to all elements (programs) of the larger set will be a powerful variable (that 

is, will account for a large percentage of the variance in the outcome measures) 

W 

for all elements of one or more subsets. 

Traditionally, researchers have sought those variables which cut across all 
possible instructional settings. These variables would apply at the same time to 
all progrms in a discovery learning model, a behavior control model, and other 
models as well. This overindulgence in parsimony may well cause the bypassing of 
those specific variables which contribute most to significant outcomes of a 
particular progrsm. 

An alternative approach to research in teacher education is proposed. Choose 
several programs from a prespecified group of instructional progrsns. The set 
may be defined in broad or narrow terms (for example, the set of first-grade reading 
progrMs, the set of objectives-based reading programs, the set of linguistic 
reading curricula for disadvantaged children, the set of inquiry-type science 
programs for primary grade students, the set of Piagetian Follow-Through programs, 
etc.). Apply the descriptive-correlational-experimental-loop research paradigm 
to individual progrmns in the set. The results of such a research program would 
reveal those variables— for each program— which are critical to affecting student 
gain in outcomes of interest. Moreover, the results wpuld reveal those dimensions 
which apply to all programs in the set. 

The Implications for teacher education of this inductive approach are clear. 
Teacher training programs i^ich emphasise general strategies for teaching may be 
providing information for the school teacher which is less than useful. Rather, 
would-be teachers could profit more from learning to teach a sample of program 
types and the behaviors crucial to each. For example, a student teacher interested 
in early childhood education could learn to implement three or four curriculum 

10/ Mo matter what the criteria chosen for the grouping and classification iof 
programs— age, grade level, subject matter, model of teaching, psychological 
orientation, etc.— it is hypothesised that this holds true. 
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progr«ns which sample a wida range of inatructional atrategiea: aay, Gilkeaon 
and Zimllea* Bank Street Progran, Buahell*a Behavior Analyaia Program, Gordon'a 
Florida Project, and Weikert'a Cognitively Oriented Approach. Or rather, if a 
more apecialised training were deaired, the aet would include only programa of a 
certain type: for example, Bngelmann'a Diatar Program; Buahell'a Behavior Analyaia 
Prograa, and Reanick*a Primary Education Project. Thua, teacher behavlora which 
are apedf ic or important to a particular program may be learned aa well aa thoae 
implementation behavlora or Inatructional actlvltlea which are generally Important 
for the aucceaa of aeveral programa. 
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